Efficient Approximate Minimum Entropy Coupling of Multiple Probability Distributions
نویسندگان
چکیده
Given a collection of probability distributions p 1 , xmlns:xlink="http://www.w3.org/1999/xlink">⋯ ,p xmlns:xlink="http://www.w3.org/1999/xlink">m the minimum entropy coupling is X ,X ( xmlns:xlink="http://www.w3.org/1999/xlink">i ~ ) with smallest H(X ). While this problem known to be NP-hard, we present an efficient algorithm for computing within 2 bits from optimal value. More precisely, construct greatest lower bound respect majorization. This construction also valid when infinite, and supports are infinite. Potential applications our results include random number generation, entropic causal inference, functional representation variables.
منابع مشابه
Epsilon Entropy of Probability Distributions
This paper summarizes recent work on the theory of epsilon entropy for probability distributions on complete separable metric spaces. The theory was conceived [3] in order to have a framework for discussing the quality of data storage and transmission systems. The concept of data source was defined in [4] as a probabilistic metric space: a complete separable metric space together with a probabi...
متن کاملProbability Distributions and Maximum Entropy
If we want to assign probabilities to an event, and see no reason for one outcome to occur more often than any other, then the events are assigned equal probabilities. This is called the principle of insufficient reason, or principle of indifference, and goes back to Laplace. If we happen to know (or learn) something about the non-uniformity of the outcomes, how should the assignment of probabi...
متن کاملApproximate probability distributions of the master equation.
Master equations are common descriptions of mesoscopic systems. Analytical solutions to these equations can rarely be obtained. We here derive an analytical approximation of the time-dependent probability distribution of the master equation using orthogonal polynomials. The solution is given in two alternative formulations: a series with continuous and a series with discrete support, both of wh...
متن کاملMaximum Entropy Beyond Selecting Probability Distributions
Traditionally, the Maximum Entropy technique is used to select a probability distribution in situations when several different probability distributions are consistent with our knowledge. In this paper, we show that this technique can be extended beyond selecting probability distributions, to explain facts, numerical values, and even types of functional dependence. 1 How Maximum Entropy Techniq...
متن کاملDetermination of Maximum Bayesian Entropy Probability Distribution
In this paper, we consider the determination methods of maximum entropy multivariate distributions with given prior under the constraints, that the marginal distributions or the marginals and covariance matrix are prescribed. Next, some numerical solutions are considered for the cases of unavailable closed form of solutions. Finally, these methods are illustrated via some numerical examples.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Information Theory
سال: 2021
ISSN: ['0018-9448', '1557-9654']
DOI: https://doi.org/10.1109/tit.2021.3076986